The present invention generally relates to a method and apparatus for improving the accuracy of quantitative images generated by multichannel imaging instruments, and more specifically, to correcting errors introduced by crosstalk between channels, with application to a broad range of imaging instruments and particularly, to flow imaging instruments using time-delay-integration image detectors.
The parallel advancement of the technology of video microscopy and techniques for preparing and staining biological samples has enabled those working in areas such as fundamental biological science, diagnostic medicine, and drug discovery to gather an ever-increasing amount of information from a single biological specimen. In the fields of cell biology and clinical cytology, for example, specimens may be stained with absorption dyes to define cell morphology, and with fluorescent dyes that attach to molecules bound to specific proteins or nucleic acid chains. Microscopes equipped for exciting and imaging the fluorescent dyes and concurrently imaging cell structures are routinely used for studying complex processes that modify cells on the gross structural level and also at the molecular level. In recent years, computational analysis of images captured from multiparameter microscopes has shown promise for automating large investigative studies such as those conducted by drug discovery and development companies and for automating complex cellular diagnostic tests for clinical medicine. Optimal use of such technology can be attained only if the signals used for image generation are accurately scaled to information about the cells being studied. However, such information can be degraded during the capture process. Specifically, interference can be introduced into a channel dedicated to a first signal due to leakage of a signal intended for a second channel. This type of signal degradation is generally referred to as channel-to-channel crosstalk.
An advancement to computer-based multiparametric imaging is disclosed in commonly assigned U.S. Patents, both entitled IMAGING AND ANALYZING PARAMETERS OF SMALL MOVING OBJECTS SUCH AS CELLS, U.S. Pat. No. 6,249,341, issued Jun. 19, 2001 (filed Jan. 24, 2000), and U.S. Pat. No. 6,211,955, issued Apr. 3, 2001 (filed Mar. 29, 2000), the complete disclosure, specification, and drawings of both of which are hereby specifically incorporated herein by reference. The technology disclosed in these applications extends the methods of computer vision to the analysis of objects either flowing in a fluid stream or moving relative to the imaging instrument on a rigid substrate, such as a glass slide. Instruments based on the inventions of the patent applications cited above deliver improved sensitivity at high spatial resolution through the use of time-delay-integration (TDI) electronic image acquisition, a method wherein signal integration is accomplished by shifting charge packets through an imaging array in synchrony with the motion of the target object being imaged.
The TDI-based flow imaging technology, with its ability to substantially improve signal-to-noise ratio, is of exceptional utility for multiparametric imaging. Each of the channels of a TDI flow imaging instrument can be dedicated to a single light source in the target objects. One such light source, for example, is the fluorescent dye attached to a molecule selected for its specificity for binding to a target protein. Each of a plurality of channels can be dedicated to a particular different dye, and all of the dyes addressed by the instrument may be present in close proximity on a single target cell. Because the dyes may have emission spectra broader than the passbands of the channels that collect their signals, channel-to-channel crosstalk can prevent the accurate estimation of the intensity of the signal from each dye.
Accordingly, it would clearly be desirable to develop a method and apparatus that simultaneously offers speed and accuracy in eliminating such channel-to-channel crosstalk. Preferably such crosstalk reduction can be achieved in conjunction with the TDI-based flow imaging method and apparatus noted above, which are intended for real time collection and processing of images from objects moving in high concentration, at high speed, through the instrument. Accordingly, the crosstalk reduction of the present invention is preferably applicable in real time and in synchrony with the collection of images of the moving targets that include indicators attached to the targets.
The present invention is directed to enabling an accurate reconstruction of information about objects imaged by an instrument using multiple channels, each channel being generally optimized to receive signals of a type differentiated from other signal types by predefined characteristics. These predefined characteristics may include, but are not limited to wavelength, a modulation of a signal received from a source, a scatter angle, a Doppler shift, and a phase shift (e.g., with respect to a reference phase). The present invention applies to, but is not limited to, instruments for collecting information from electromagnetic waves in all portions of the spectrum, by acoustic waves, by particle flux, and by measurement of object characteristics such as conductivity, chemical reactivity, size, shape, and mass.
One example of an application of the present invention is its use in a multiple-wavelength optical imaging instrument. In such an instrument, each channel is made sensitive to electromagnetic radiation of wavelengths bounded by an upper and lower limit, defining different wavebands for each channel. Typically these limits are determined by the characteristics of one or more filters disposed in a path between a light source and a photodetector servicing a channel. The images in each channel are detected, producing signals that are processed by the present invention to correct errors in alignment between the channels and a reference and then, to correct for crosstalk between the channels.
Thus, the present invention is directed to a method and apparatus that not only corrects for crosstalk between channels, but also ensures that signal data in each channel is properly aligned with signal data in other channels, so that the benefit from the crosstalk correction is not degraded by signal misalignment.
In one preferred embodiment, a method is provided for correcting signal misalignment between individual channels in a multichannel imaging system, such that data in a first channel is substantially aligned with data in other channels. The method also includes the step of reducing erroneous contributions to signal data from a source intended to provide signal data for other channels.
Preferably, the signal data are used to produce an image for display. Accordingly, a preferred embodiment is directed to a method that includes the step of spatially aligning images input in an image ensemble from a plurality of channels, such that each image in the image ensemble is substantially aligned with other images in the image ensemble, and the step of applying spectral crosstalk corrections, to remove the channel-to-channel crosstalk from the image ensemble output.
In one embodiment, the step of spatially aligning images includes the step of utilizing two classes of information, including a first and second class of constants. The first class of constants includes horizontal and vertical spatial offsets, which are derived from an on-line calibration image. The second class of constants is accessed during the step of spatially aligning images, but is not modified. Preferably the second class of constants includes at least one of channel start columns for each image, and inverted source coefficients.
The horizontal and vertical spatial offsets are preferably generated based upon a comparison of each image in an image ensemble with a calibration image. The comparison with a calibration image can be performed when a system for generating the multichannel signal is initialized, and/or periodically during the use of a system for generating the multichannel signal.
The step of generating the horizontal and vertical spatial offsets can include the steps of detecting a boundary of an image, preparing a correlogram based on the boundary and a reference image, determining a peak of the correlogram, and repositioning the image to correspond to a pixel closest to the peak of the correlogram.
Preferably the horizontal and vertical spatial offsets are determined for each pixel of the image, and the detection of the boundary of an image involves the use of a two-dimensional gradient operator to suppress flat surfaces and to enhance object boundaries. In one embodiment, preparing the correlogram based on the boundary and the reference image involves preparing a correlogram in the spatial domain, while in other embodiments the correlogram is prepared in the frequency domain.
In an embodiment in which the correlogram is prepared in the spatial domain, a Fourier Transform is performed on boundary data for the image and the reference image. Those results are multiplied to generate a product, and an inverse Fourier Transform is performed on that product.
To prepare the correlogram based on the boundary and the reference image in the frequency domain, first a Fourier Transform is performed on the boundary data for the image and the reference image. Then a conjugation operation is applied to one of the results of the Fourier Transforms. Next, the result of the conjugation operation is multiplied with the boundary data for the image to generate a product, and an inverse Fourier Transform is performed on the product.
To minimize errors, groups of images in each data channel are preferably processed together, such that a cumulative correlogram is generated for each data channel.
Once the correlogram is complete, the peak of the correlogram defines the aligned position of the image, relative to the reference image employed. The peak of the correlogram is determined by employing a Taylor series expansion, eigenvalues and eigenvectors. The image is then manipulated to align, to the nearest pixel, with that peak. Then, the image is reconstructed by interpolating to a fraction of a pixel, to align within a fraction of a pixel, with the true peak of the correlogram. Preferably, the interpolation involves the step of applying a two-dimensional interpolation.
In one embodiment, the step of applying a two-dimensional interpolation includes the step of computing a new amplitude value for each pixel based on a weighted sum of a group of surrounding pixels. Preferably, the weighted sum is determined by a Taylor series expansion based on a group of nine pixels, eight pixels of which surround a common origin pixel. Coefficients are applied to each pixel value, and the sum of a matrix of the coefficients is equal to 1.0.
The step of reducing erroneous contributions to that channel""s measurement by sources intended for other channels preferably involves solving a set of linear equations relating source values to measurement values, wherein each channel is represented by a linear equation. It is also preferred that the set of linear equations are solved for each pixel in each image in each channel. The set of linear equations relating source values to measurement values can be solved by applying singular value decomposition to a matrix form of the set of linear equations.
The signal data, and/or corresponding images, can be spatially aligned in real-time. After the spatial alignment is completed, spectral crosstalk corrections can also be applied in real-time, or after the signal data/images have been stored for a period of time. The signal data/images can also be spatially aligned after having been stored for a period of time.
Another aspect of the present invention relates to a method for correcting errors in a multichannel imaging system, wherein each channel is intended to contain signal information relating to an image of an object that has been produced by only one type of source. The method involves focusing light from the object along a collection path, and dispersing the light that is traveling along the collection path into a plurality of light beams, such that each light beam corresponds to a single source. Each of the light beams is then focused to produce a respective image corresponding to that light beam. A detector is provided, disposed to receive the respective images. The detector generates an output signal corresponding to each image. For each output signal, a signal alignment correction and a crosstalk correction are applied.
In addition to the aforementioned embodiments relating to the method, the present invention also relates to a system having elements that carry out functions generally consistent with the steps of the method described above. One such system relates to a multichannel imaging system for generating an ensemble of images from an object for each field of view, wherein each image in the ensemble contains information from substantially only one type of source. The system includes a collection lens disposed so that light traveling from the object passes through the collection lens and travels along a collection path, and a dispersing component disposed in the collection path so as to receive the light that has passed through the collection lens, dispersing the light into a plurality of separate light beams, each light beam being directed away from the dispersing component in a different predetermined direction. The system also includes an imaging lens disposed to receive the light beams from the dispersing component, thereby producing the ensemble of images. The ensemble comprises a plurality of images corresponding to each of the light beams, each image being projected by the imaging lens toward a different predetermined location. A multichannel detector is disposed to receive the plurality of images produced by the imaging lens, and produces a plurality of output signals, such that a separate output signal is produced for each of the separate light beams. Finally, the system includes means for processing each output signal, wherein the means performs the functions of correcting output signal misalignment between individual channels, such that an image generated by an output signal in each channel is substantially aligned with a corresponding image in each other channel, reducing erroneous contributions to that channel""s measurement by sources intended for other channels.
The system also preferably includes a display electrically coupled to the means, the display generating an image for each output signal as modified by the means. The means for processing preferably includes a memory in which a plurality of machine instructions defining a signal conditioning application are stored, and a processor that is coupled to the memory to access the machine instructions, and to the display. Execution of the machine instructions by the processor cause it to spatially align images that are displayed, based on the output signals from the multichannel detector, such that each image is substantially aligned with other images. The processor also applies the spectral crosstalk corrections, to remove the channel-to-channel crosstalk from the displayed images.
It is further contemplated that the means for processing the signal alternatively comprise a programmed computer, an application specific integrated circuit, or an oscilloscope.
Yet another embodiment of the system includes a plurality of different detectors, such that an image corresponding to a different source is directed to each detector, and the plurality of different detectors collectively comprise the multiple channels. The detectors employed in this embodiment of the system are preferably pixilated. For example, a TDI detector can beneficially be employed to produce output signals by integrating light from at least a portion of an object over time, while relative movement between the object and the imaging system occurs.